Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[WIP] Complex pushdown #402

Closed
wants to merge 18 commits into from
Closed

[WIP] Complex pushdown #402

wants to merge 18 commits into from

Conversation

martint
Copy link
Member

@martint martint commented Mar 7, 2019

These are the initial steps towards #18. It includes the prerequisite cleanups and refactorings (#363) and the beginnings of arbitrary filter pushdown. Just want to get it out there so that people can see where we're going with this.

Depends on #363

@cla-bot cla-bot bot added the cla-signed label Mar 7, 2019
@martint martint force-pushed the pushdown branch 3 times, most recently from 1f0b334 to 97a8ec0 Compare March 7, 2019 04:19
martint added 18 commits March 7, 2019 09:47
More accurately, constructs a set of alternate plans with the
filter pushed into the table scan based on available table layouts.
This test relies on a layout being picked during planning. For
LocalQueryRunner this only happens because of a synthetic rule
(PickLayout w/o predicate) that attaches a layout to the table scan.
So, in a sense, it's just a coincidence that it works.

In distributed execution, the job is done by AddExchanges, so
we want to make sure we're testing that behavior.
It's just selecting a layout for the raw table scan, so
no need to go through the logic for pushing a predicate,
etc.
This change hides table layouts from the engine as a first-class
concept. We keep the SPI as is for backward compatibility for now.

When predicates are pushed into a table scan by PickLayout (now
PushPredicateIntoTableScan) or AddExchanges, we now replace the
table handle associated with the table scan with a new one that
contains the reference to the ConnectorTableLayoutHandle under
the covers.
It now contains a single rule, so no point in
having it return a rule set.
In order to translate expression to row expressions, the
code was first replacing all symbol references with field
references for the corresponding ordinal inputs.

This is unnecessary, as the translation can be done on demand
as the expression is translated to a row expression.
The inferred type of the former expression is INTEGER, which
doesn't match the signature of combineHash function call.
They were only being used in tests. The engine no longer
relies on them for query execution.
There's no longer a conflict with analyzeExpressionsWithInputs so
simplify the name
There's only one caller, so no need for an extra indirection.
The new class is to facilitate obtaining the type of an expression and its subexpressions
during planning (i.e., when interacting with IR expression) and to remove spurious
dependencies on the SQL parser. It will eventually get removed when we split the AST
from the IR and we encode the type directly into IR expressions.
@jerryleooo
Copy link
Member

I guess #7994 is the successor of this PR?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Development

Successfully merging this pull request may close these issues.

3 participants